\section{The Experimental Setup} \label{experimental-setup}

%\emph{Explaining the experiment, the goal, the order and role of each room and phase \\}

We carried out an experiment in a virtual world specially designed to test our hypothesis. The 3D view of a room in the virtual world is shown in Figure~\ref{object-room}. The virtual world was designed using the GIVE platform~\cite{Koller_Striegnitz_Gargett_Byron_Cassell_Dale_Moore_Oberlander_2008}. The GIVE platform includes a set of tools for designing virtual houses that can contain different virtual objects with distinguishing properties (such as color, shape, etc). 

\begin{figure}[h!]
\includegraphics[width=7.5cm]{object.png} 
\caption{Screenshot of the 3D virtual environment used by the subjects. Navigation instructions were given in Spanish. \label{object-room}}
\end{figure}

We recruited fifty subjects; all of them were native speakers of Spanish and had no previous knowledge of Russian. The subjects completed the experiment in an average time of 12.5 min each. 20\% of the participants were women and the rest were men. Most of them were university students from different majors (computer science, psychology, biology, etc). The average age was 27.
Before they started the experiment, we told the subjects that they were going to go through a short language-learning 3D video game to learn some vocabulary in Russian. The words they learned were from 3 categories: Taxonomical (3 names of objects), Absolute (4 colours) and Relative (2 sides).  Of the 50 subjects, we made two equal groups of 25 each. The MR (Minimal Reference) group received vocabulary exercises with minimal REs regarding objects in context (i.e. just enough information to successfully identify the target object), whereas the OR (Overspecified Reference) group received exercises with overspecified REs. 

%\subsection {The Virtual World}
%\emph{
%First: Description of our world \\
%Second: Advantages of using a Virtual world \\
%Should we talk about saliency ? \\}

For the purpose of this experiment, we implemented an automated system which uses the GIVE platform to generate referring expressions to describe the virtual world objects in Russian. The system helps the subjects navigate through the virtual world while teaching them a series of Russian words. Instructions on how to interact with virtual objects and when to move to the next phase of the experiment were given in Spanish. All referring expressions to objects the subject had to identify were given in Russian. During the experiment, the subjects went through five phases in the virtual world sequentially as described below.

The subject's first contact with the virtual world was in the \emph{Tutorial Phase}: they learned how to navigate the world and the conventions of the experiment. We explained that identifying an object was done by pressing the button on its right. 

Next came the \emph{Priming Phase} (which includes the Object, Color and Spatial Relation Rooms), where the subjects were primed with the Russian vocabulary (colors, objects, and orientations). Figure~\ref{object-room} shows the view that a subject perceived at the beginning of the Learning Phase of the experiment. 
During the \emph{Learning Phase}, we asked the subjects to press the buttons beside the objects in a room and gave them the name of the object in Russian. Each word stayed visible for 5 seconds and could only be received once (subsequent pressings did not give the word again). No specific order of the objects was enforced. The door to the next room only opened when all buttons in the current room were pressed. 

The \emph{First Test Phase} evaluated how well the subjects remembered the vocabulary primed in the Priming Phase. Subjects were given a word in Russian and asked to press the button corresponding to that object or property, for a total of 7 objects. The REs produced in this phase were minimal for both the MR group and the OR group. In order to resolve them, the subjects needed to remember all the words.  

In the \emph{Exercise Phase} subjects were given exercises to practice identifying complex referential expressions (combinations of the 4 colors and 3 objects, for a total of 12 objects). The REs given differed for the MR and OR subjects. 
The MR subjects received an Absolute property (color) and a Taxonomical property (object), e.g. \emph{zoltii stul} (``yellow chair" in Russian). 
The OR subjects received an Absolute property, a Taxonomical property and a Relative property (location of the object with regards to a neighboring object), e.g. \emph{zoltii stul sleva ot krasnii svet} (``yellow chair on the left of the red light"). 

\begin{figure}[h!]
\includegraphics[width=7.5cm]{minspec.png} 
\caption{Exercise Phase:  Referring expression received by a subject in the MR condition: \emph{zoltii stul} means `yellow chair'. \label{minimal}}
\end{figure}


Figure~\ref{minimal} shows a RE as received by a subject in the MR condition. In this situation, the target referent, the red plant, was visible to the subject. Figure~\ref{overspecification} shows a RE as received by a subject in the OR condition. The RE in Figure~\ref{overspecification} is overspecified since there was only one red plant in the room. In the moment the screenshot was captured, the target referent was not visible to the subject. The REs in both figures refer to the same object in the virtual world; however, Figure~\ref{minimal} gives only the minimal semantic content needed needed to find the object, whereas Figure~\ref{overspecification} gives an overspecified description.  We decided to overspecify the REs---such as \emph{yellow chair}---with a relation to a neighbouring object---\emph{to the left of the red light}---since there are case studies that show that this is the preferred property that is most frequently overspecified in corpora~\cite{Viethen_2008}. 

\begin{figure}[t]
\includegraphics[width=7.5cm]{overspec.png} 
\caption{Exercise Phase:  Referring expression received by a subject in the OR condition \emph{zoltii stul sleva ot krasnii svet} means `yellow chair on the left of the red light'. \label{overspecification}}
\end{figure}

The \emph{Second Test Phase}, with an identical procedure to the first one, existed in order to be able to compare the subjects' relative improvement after the Exercise Phase. 

The advantages of using a virtual environment for the experiment are several. First, it was easy to build the world and tailor it according to our needs. Moreover, it had an added element of interactivity as opposed to flashcards. Finally, it gave us an array of useful information, such as logging objects visible to the subject as he moved in the virtual world. 

Since our subjects could not read the Cyrillic alphabet, we transliterated the REs in Russian using the ISO 9:1995 standard Romanization of Cyrillic, with the phonetic concession to Spanish speakers of replacing `y' with `i' and `zh' with `z'.



